语音触发检测是一项重要的任务，它可以在目标用户说关键字短语时激活语音助手。通常对探测器进行语音数据培训，独立于说话者信息，并用于语音触发检测任务。但是，这样的说话者独立语音触发探测器通常会遭受绩效降低，因为代表性不足的群体，例如重音说话者。在这项工作中，我们提出了一个新颖的语音触发探测器，该触发探测器可以使用目标扬声器中的少量话语来提高检测准确性。我们提出的模型采用编码器架构。尽管编码器执行扬声器独立语音触发检测，但类似于传统检测器，解码器预测了每种话语的个性化嵌入。然后，获得个性化的语音触发分数作为在注册话语的嵌入与测试话语之间的相似性得分。个性化的嵌入允许在计算语音触发评分时适应目标扬声器的语音，从而提高语音触发检测精度。实验结果表明，与基线扬声器独立语音触发模型相比，所提出的方法相对降低（FRR）的相对降低38％。
translated by 谷歌翻译
Intelligent agents have great potential as facilitators of group conversation among older adults. However, little is known about how to design agents for this purpose and user group, especially in terms of agent embodiment. To this end, we conducted a mixed methods study of older adults' reactions to voice and body in a group conversation facilitation agent. Two agent forms with the same underlying artificial intelligence (AI) and voice system were compared: a humanoid robot and a voice assistant. One preliminary study (total n=24) and one experimental study comparing voice and body morphologies (n=36) were conducted with older adults and an experienced human facilitator. Findings revealed that the artificiality of the agent, regardless of its form, was beneficial for the socially uncomfortable task of conversation facilitation. Even so, talkative personality types had a poorer experience with the "bodied" robot version. Design implications and supplementary reactions, especially to agent voice, are also discussed.
translated by 谷歌翻译
Generative models, particularly GANs, have been utilized for image editing. Although GAN-based methods perform well on generating reasonable contents aligned with the user's intentions, they struggle to strictly preserve the contents outside the editing region. To address this issue, we use diffusion models instead of GANs and propose a novel image-editing method, based on pixel-wise guidance. Specifically, we first train pixel-classifiers with few annotated data and then estimate the semantic segmentation map of a target image. Users then manipulate the map to instruct how the image is to be edited. The diffusion model generates an edited image via guidance by pixel-wise classifiers, such that the resultant image aligns with the manipulated map. As the guidance is conducted pixel-wise, the proposed method can create reasonable contents in the editing region while preserving the contents outside this region. The experimental results validate the advantages of the proposed method both quantitatively and qualitatively.
translated by 谷歌翻译
Collecting sufficient labeled data for spoken language understanding (SLU) is expensive and time-consuming. Recent studies achieved promising results by using pre-trained models in low-resource scenarios. Inspired by this, we aim to ask: which (if any) pre-training strategies can improve performance across SLU benchmarks? To answer this question, we employ four types of pre-trained models and their combinations for SLU. We leverage self-supervised speech and language models (LM) pre-trained on large quantities of unpaired data to extract strong speech and text representations. We also explore using supervised models pre-trained on larger external automatic speech recognition (ASR) or SLU corpora. We conduct extensive experiments on the SLU Evaluation (SLUE) benchmark and observe self-supervised pre-trained models to be more powerful, with pre-trained LM and speech models being most beneficial for the Sentiment Analysis and Named Entity Recognition task, respectively.
translated by 谷歌翻译
Self-supervised learning (SSL) methods such as WavLM have shown promising speech separation (SS) results in small-scale simulation-based experiments. In this work, we extend the exploration of the SSL-based SS by massively scaling up both the pre-training data (more than 300K hours) and fine-tuning data (10K hours). We also investigate various techniques to efficiently integrate the pre-trained model with the SS network under a limited computation budget, including a low frame rate SSL model training setup and a fine-tuning scheme using only the part of the pre-trained model. Compared with a supervised baseline and the WavLM-based SS model using feature embeddings obtained with the previously released 94K hours trained WavLM, our proposed model obtains 15.9% and 11.2% of relative word error rate (WER) reductions, respectively, for a simulated far-field speech mixture test set. For conversation transcription on real meeting recordings using continuous speech separation, the proposed model achieves 6.8% and 10.6% of relative WER reductions over the purely supervised baseline on AMI and ICSI evaluation sets, respectively, while reducing the computational cost by 38%.
translated by 谷歌翻译
We present the first neural network model to achieve real-time and streaming target sound extraction. To accomplish this, we propose Waveformer, an encoder-decoder architecture with a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder. This hybrid architecture uses dilated causal convolutions for processing large receptive fields in a computationally efficient manner, while also benefiting from the performance transformer-based architectures provide. Our evaluations show as much as 2.2-3.3 dB improvement in SI-SNRi compared to the prior models for this task while having a 1.2-4x smaller model size and a 1.5-2x lower runtime. Open-source code and datasets: https://github.com/vb000/Waveformer
translated by 谷歌翻译
给定数千种同样准确的机器学习（ML）模型，用户如何在其中选择？最近的ML技术使领域专家和数据科学家能够为稀疏决策树生成完整的Rashomon设置，这是一套几乎最理想的可解释的ML模型。为了帮助ML从业者识别具有此Rashomon集合中理想属性的模型，我们开发了Timbertrek，这是第一个交互式可视化系统，该系统总结了数千个稀疏决策树的规模。两种用法方案突出了Timbertrek如何使用户能够轻松探索，比较和策划与域知识和价值观保持一致的模型。我们的开源工具直接在用户的计算笔记本和Web浏览器中运行，从而降低了创建更负责任的ML模型的障碍。Timbertrek可在以下公共演示链接中获得：https：//poloclub.github.io/timbertrek。
translated by 谷歌翻译
开发准确，灵活和数值有效的不确定性量化（UQ）方法是机器学习中的基本挑战之一。以前，已经提出了一种名为Disco Nets的UQ方法（Bouchacourt等，2016），该方法通过最大程度地减少训练数据中所谓的能量评分来训练神经网络。该方法在计算机视觉中的手姿势估计任务上表现出了出色的性能，但是尚不清楚该方法是否可以很好地对表格数据进行回归，以及它如何与较新的高级UQ方法（例如NGBOOST）竞争。在本文中，我们提出了改进的迪斯科网络神经结构，该建筑接受了更稳定和平稳的训练。我们将这种方法基于其他现实世界表格数据集，并确认它具有竞争力甚至优于标准的UQ基准。我们还为使用能量评分学习预测分布的有效性提供了新的基本证明。此外，我们指出的是，迪斯科的原始形式忽略了认知的不确定性，只捕获了不确定性。我们为这个问题提出了一个简单的解决方案。
translated by 谷歌翻译
在任何给定的机器学习问题中，可能有许多模型可以很好地解释数据。但是，大多数学习算法仅返回这些模型中的一种，使从业者没有实用的方法来探索替代模型，这些模型可能具有超出损失函数中可以表达的内容的理想属性。 Rashomon集是所有这些几乎最佳模型的集合。 Rashomon集可能非常复杂，尤其是对于高度非线性功能类，允许复杂的交互项，例如决策树。我们提供了第一种完全列举稀疏决策树的Rashomon设置的技术；实际上，我们的工作提供了针对高度非线性离散功能类别的非平凡问题的所有Rashomon设置的首次列举。这使用户可以在所有近似同样好的模型中对模型选择的前所未有的控制水平。我们在专门的数据结构中表示Rashomon集，该数据结构支持有效的查询和采样。我们显示了Rashomon集的三个应用：1）它可用于研究一组几乎最佳树的重要性（与一棵树相对），2）Rashomon设置的精确度使Rashomon集可以枚举Rashomon集合。平衡的精度和F1得分，以及3）完整数据集的Rashomon集可以用于生产仅使用数据集的子集构建的Rashomon集。因此，我们能够检查新镜头问题的Rashomon集合，使用户能够选择模型，而不是受到仅产生单个模型的算法的摆布。
translated by 谷歌翻译
我们在随机多臂匪徒问题中使用固定预算和上下文（协变）信息研究最佳武器识别。在观察上下文信息之后，在每一轮中，我们使用过去的观察和当前上下文选择一个治疗臂。我们的目标是确定最好的治疗组，这是一个在上下文分布中被边缘化的最大预期奖励的治疗组，而错误识别的可能性最小。首先，我们为此问题得出半参数的下限，在这里我们将最佳和次优的治疗臂的预期奖励之间的差距视为感兴趣的参数，以及所有其他参数，例如在上下文中的预期奖励，作为滋扰参数。然后，我们开发“上下文RS-AIPW策略”，该策略由随机采样（RS）规则组成，跟踪目标分配比和使用增强反向概率加权（AIPW）估算器的建议规则。我们提出的上下文RS-AIPW策略是最佳的，因为错误识别概率的上限与预算到Infinity时的半参数下限相匹配，并且差距趋于零。
translated by 谷歌翻译